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ABSTRACT 

A study assessed whether means for student 
evaluations of instruction tended to be lower in research methods 
classes than in other communication classes. Questionnaires were sent 
to 126 communication departments listed in the 1993 Speech 
Communication Association Directory and which were thought likely to 
require undergraduate research methods courses. A total of 77 
questionnaires were completed, for a response rate of 61%, Results 
indicated that (1) student evaluations of communication research 
methods courses were lower than those for other communication 
classes; (2) courses receiving relatively low evaluations were more 
likely to be larger, intended for less experienced students, and 
geared more toward methods rather than statistics compared to classes 
rated as relatively high or average; (3) the number of elective 
courses offered per year was lower than the number of required 
courses; and (4) the mean class size for elective courses was 21.43 
while the mean for required courses was 53.92. Findings suggest that 
if a communication department requiring a research methods course 
wants to counteract biases in student evaluations, it should keep 
class size to less than 40 and intend the class for juniors and 
seniors . (RS) 
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Charles Pavitt 

Student Evaluations of Communication Research Methods 
Courses: Relative Levels and Contributing Factors 

O 

in 

CS More than a thousand papers have been published concerning the 

N» 

CO 

q strengths and weaknesses of student evaluations of instruction at 
UJ 

the university and college level (see Costin, Greenough, & Menges, 
1971; and Marsh, 1984, for important literature reviews) . Although 
commentators make generally positive assessments of student 
evaluations, research has revealed several biases that can affect 
the validity of summary indices for questions employing numerical 
• scales. Evaluation item means can be affected by factors such as 
student motivation, class size, and course content. In 
universities in which summary indices are an integral part of the 
evaluation of instruction, these biases can adversely affect an 
instructor's apparent performance level and, in turn, their 
concrete rewards for performance, such as merit pay, promotion, and 
tenure . 

As established in Frey and Botan's (1988) survey, many 
communication departments have included classes in research methods 
as requirements in their undergraduate programs. Communication 
research methods classes may be particularly susceptible to bias in 
summary indices of student evaluations. Courses involving 
mathematics characteristically receive lower evaluations than 
q humanities and social science courses, and the inclusion of a 
v9 mathematics -based class in a communication program may result m 
o lower student motivation and relatively low student evaluations 
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The primary purpose of the present study is to assess whether 
means for student evaluations of instruction tend to be lower in 
research methods classes than in other communication classes. The 
indication of such a tendency may aid universities in the 
interpretation of these data and in insuring the equitable 
distribution of concrete rewards for skillful instruction. A 
secondary purpose of the study is to determine whether a set of 
factors consisting of course level, content coverage, class size, 
the number of sections offered, and the presence of teaching 
assistants are associated with relative evaluations of 
communication research methods courses. 

Methods 

A questionnaire, developed by the author with the aid of other 
faculty for the purposes just described, was sent to 126 
communication departments listed in the 1993 SCA Directory that the 
author identified as likely to require undergraduate research 
methods courses. The sample included, among others, all 
departments noted in recent reviews (Edwards, Watson, & Barker, 
1988; Trost, Barker, and Barker, 1988) as having highly regarded 
doctoral or master's programs, and all departments at state 
university main campuses. 

Results 

Overview 

We received 77 responses to our mailing, a return rate of 
61.1%. Of these 77, 49 (63.6%) required research methods for at 
least some of their undergraduates, 9 (11.7%) offered research 
methods as an elective (11.7%), and 19 (24.7%) did not offer the 
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course . 
Required 

Of the 49 responding departments that required research 
methods for some or all undergraduates, two (4.1%) reported that 
student evaluations for the class were very high, five (10 . 2%) 
reported eveluations to be above average, 12 (24 . 5%) reported 
evaluations to be average, 18 (36.7%) reported evaluations to be 
below average, and six (12.2%) reported evaluations to be very low, 
relative to other communication courses. Two (4.1%) reported that 
evaluations were dependent on the instructor, and four (8.2%) did 
not answer this question. These data suggest that student 
evaluations of research methods courses are lower than those for 
other communication classes. 

Three of the factors included in the study (course level, 
class size, and content coverage) were found to be associated with 
relative level of evaluations. The results for these factors are 
best described by dividing the responses into two categories; one 
including classes evaluated above or equal to the average 
communication course (n = 19) , and the other including classes 
evaluated below average (n « 24) . For course level, only one class 
evaluated above or equal to average was designed for first and 
second year students (5.3%), whereas eight classes evaluated below 
average were similarly intended (33.3%) . For class size, the mean 
enrollment for classes evaluated above or equal to average was 
35 . 32 , while the mean for classes evaluated below average was 
68.65. For content coverage, classes evaluated above or equal to 
average spent mean proportions of 35.47% of the time on statistics 
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and 51.84% of the time on research methods (such as experiments, 
surveys, and content analysis) , with the analogous proportions for 
classes evaluated below average at 26.10% and 57.64%. Thus, 
communication research methods courses receiving relatively low 
evaluations were more likely to be larger, intended for less 
experienced students, and geared more toward methods rather than 
statistics, when compared with classes rated as relatively high or 
average . 

The other two factors explored in the study were not 
associated with relative level of student evaluations. Programs 
whose methods courses were evaluated above or equal to average 
offered a mean of 2.68 sections per year, as compared to 2.85 for 
programs whose methods courses were evaluated as below average. 
Although classes rated as below average had a greater mean number 
of teaching assistants (1.25 versus .58), this difference is an 
artifact of class size; the 26 classes with no TAs averaged 33.17 
students, the nine classes with one TA averaged 54.56 students, and 
the eight classes with more than one TA averaged 110.63 students. 
A three way contingency table showed that the presence or absence 
of TAs had no effect on relative course evaluations dependent on 
whether the class has more or less than 40 students. 
Elective 

Of the nine respondents who indicated that their program 
offers research methods as an elective, 3 (33.3%) reported that 
student evaluations were above average, 4 (44.4%) reported 
evaluations to be below average, and 2 (22.2%) did not answer the 
question. Given these small numbers, meaningful comparisons 
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between these* categories are not possible. 

A comparison between the required and elective data sets 
reveal some interesting findings. The overall proportion of time 
spent on statistics (26.07%) and research methods (48.93*) in 
elective courses was similar to the overall proportions in required 
classes (30.24% and 55.08% respectively). There were, however, 
striking differences between the two samples in the mean number of 
sections offered per year (elective, 1.07; required, 2.78) and the 
mean class sizes (elective, 21.43; required, 53.92). 

Discussion 

The results of this study are consistent with past research on 
biases in student eveluations. Mathematics -based courses are 
generally evaluated lower than social science and humanities 
courses (Marsh, 1984) . Class size is known to have a negative 
impact on evaluations (Costin, Greenough, & Menges, 1971) . The 
finding that classes with a greater proportion of time spent on 
statistics, rather than methods, may be consistent with past 
findings that course difficulty is actually positively related with 
evaluations (Marsh, 1980) . The effect for course level also makes 
sense if we can assume that upper-class students are more motivated 
to take a communication research methods class than lower-class 
students. Unfortunately, the sample of respondents whose programs 
offer research methods as an elective was too small to allow for 
meaningful analyses. Based on past research, the expectation would 
be for those classes to be evaluated more positively than required 
classes (Marsh, 1980) . 

These results suggest that if a communication department 
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requiring a research methods course wants to counteract biases in 
student evaluations of these courses, it should keep class sizes at 
less than 40, and intend the class for juniors and seniors. If the 
needs of the program make these recommendations impossible, than 
the department should expect student evaluations in these courses 
to be below average, should include these factors in their criteria 
when evaluating the instructor's performance, and should attempt to 
educate other members of the university community involved in the 
evaluation of communication faculty to strongly consider these 
factors in their judgments. 
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